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Two studies were conducted in an effort to better 
understand the role of digitized speech as feedback in computer 
assisted instruction (CAI) . The first study examined the use of 
familiar and unfamiliar voice feedback in two CAI lessons designed to 
teach advertising techniques to fifth graders. Subjects were 100 
fifth graders from suburban Minneapolis (Minnesota). No differences 
were found between responses to the Tamiliar and unfamiliar voices. 
In the second study, 145 sixth graders and a subset of 41 high and 49 
low reading ability students from suburban Minneapolis (Minnesota) 
listened to feedback that was spoken audio only, printed and spoken, 
and spoken by an animated character. Analysis indicates that the type 
of feedback makes no significant difference, although there is a 
consistent tendency for students receiving the voice only feedback to 
score higher on the posttest, possibly because supplementing with 
text or graphics divides learners' attention. Overall results suggest 
that whose voice is used seems to be of little importance, although 
the gender of the speaker can influence performance. Females 
outperformed males when a female voice was used. When the agent was 
male, no effect for gender was found. Students appeared to enjoy the 
animated characters and to talk back to the computer more than did 
students in other treatments. Nine figures and four tables illustrate 
the findings. (Contains 18 references.) (SLD) 



>'c 1c 1c 1c 1c * 1: i'c 1c 1c 1c 1c 1c * 1c 1c * 1c 1c 1c 1c 1c 1c Vc 1c * 1c 1c 1c . 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c Vc 1c 1: 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 

Reproductions supplied by EDRS are the best that can be made * 
from the original document. * 

1c 1c 1c 1c iV 1c 1c 1c 1c 1c 1c * 1c 1c 1c 1c 1c 1c 1c 1c :V 1c 1c 1c 1; ft 1c ft It 1< 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 1c 



U.S. DEPA*TMEN T Of EOUCATJON 
OHice of Educational Research and improvement 
EDUCATIONAL RESOURCES INFORMATION 



O Th.s document has been reproduced as 
received irom the person or organization 
originating it 

O Minor changes have been made to improve 
reproduction quatity 



a Points ot view or opinions stated m this docu- 
ment do not necessarily represent official 
OERI position or policy 



Digitized Speech as Feedback 
in Computer-Based Instruction 



Authors: 

Gregory C. Sales 
Michael D. Johnston 



CENTER (ERIC) 



Tide: 



2 * 



• PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 

Michael Slmonson 



851 




Digitized Speech as Feedback 
AECT 1993 2 



Digitized Speech as Feedback 
in Computer-Based Instruction 

We should not be surprised by the flood of educational products utilizing 
digitized speech currently moving through the home and school markets. From 
the time of Socrates, speech has been an important part of instruction. For 
most of history speech has been the primary mean of communicating content 
to the learner. Only recently has text in the form of books and other 
technologies been effectively utilized to increase the number of learners that 
can be reached and to provide a consistent delivery of information. Naturally 
then, the integration of speech into instructional software has great intuitive 
appeal, especially if it can be linked to text, graphics and animation. 

While the phenomenon of talking software may now be upon us in full, it 
has been evolving for years. Several techniques for generating computer 
speech have existed since the 1970s. However, these early techniques were 
seldom utilized because of cost, equipment requirements or quality issues. 
Recent advances in hardware and software have made the use of digitized 
speech a practical option for many instructional applications. 

Software designers and developers appear to believe that speech will make 
computer-based instruction (CBI) more attractive, more effective, and 
ultimately more marketable. They are rushing to integrate speech into a 
variety of products. Their motives for applying this technology, however, may 
be based purely on consumers 1 demonstrated eagerness to see, use and own 
the latest innovations. 

There is little empirical research to indicate that educational gains can be 
achieved by simply inc( rporating speech in computer software. Decades of 
empirical research on t!ie use of sound, music and speech in instruction 
(Brophy, 1981; Chiang, 1983; Cumming & McCorriston, 1981; Fleming, 1980; 
Haugh, 1952; Rulon & Others, 1943; Rysavy & Sales, 1991; Sanders, 
Benbasset & Smith, 1976) provide only limited guidance on how these features 
might be used in computer-based instruction. This guidance falls well short of 
identifying when speech is an appropriate component of CBI, the role it should 
perform, or the interface and operational support that a program should 
provide to ensure optimal learning. 

Many questions that may inform decision about the utilization of speech in 
software remain unanswered. This paper reports on two studies conducted in 
an effort to help use better understand the role of digitized speech as feedback 
in CBI (A third study in this line of research is being reported at this meeting 
by Michael D. Johnston.) 

Study 1 . The first study examined the use of familiar and unfamiliar voice 
feedback in two computer-based lessons designed to teach advertising 
techniques (propaganda techniques) to fifth graders. The role of the voice 
feedback was to provide knowledge of response, guidance and motivation during 
the practice portion of the instruction. (See figures 1-6 for examples of the 
screen displays from the computer-based lesson, the feedback types, and 
content of the feedback messages.) 

The primary research questions addressed in this study were: 
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• Will the use of a feedback provider whom learners know influence their 
performance in the instruction, as measured by the posttest? 

• Will the use of (1) voice with text or (2) voice with a speaker's image 
influence the learners' performance in computer-based instruction? 

• Are there differences (e.g., use of optional buttons, achievement, time 
to complete) in performance that can be attributed to gender? 

. Study 2 . The second study examined the effects of three types of feedback 
(1) spoken audio only, (2) printed and read (spoken), and (3) spoken by an 
animated character. These feedback types were substituted in the 
propaganda software used for the first study. (See figures 7-9 for examples of 
the screen displays from the computer-based lesson, the feedback types, and 
content of the feedback messages.) 

The primary research questions were: 

• Will the type of feedback (i.e., form of presentation - (1) spoken audio 
only, (2) printed and read (spoken), and (3) spoken by an animated 
character) effect performance? 

• Will reading ability interact with performance as measured by 
posttest scores? 

Literature Review 

What is the role of speech in instruction? It would be difficult to deny that 
speech can play a central role in the delivery of instruction. Most modern 
models of instruction assume the extensive use of spoken or written words. 
For example, each of the events of instruction (Gagne, Briggs & Wager, 1992), 
from informing the learner of the objectives to enhancing retention and 
transfer, can be accomplished at some level through spoken words. 

The power and influence of the spoken word is equally difficult to deny. The 
effectiveness and impact of speech in such roles as the delivery of content, 
feedback and motivation in classroom instruction are well documented (e.g., 
Brophy, 1981, 1986). 

Salomon (1979, 1985) argues that research on the effectiveness of media 
must look at the most essential characteristics of symbol systems being 
utilized. In software that utilizes digitized speech, Brophy's work on the impact 
of teachers' comments indicates that these essential characteristics are the 
familiarity of the source, the nature of the learners relationship with the 
source, and the attitude communicated through tones and inflections in 
feedback to the learner. 

What do we know about the effects of speech (and other audio) in computer 
software? Researchers have expressed concern over techniques and strategies 
for the use of speech in CBL Sanders, Benbasset and Smith (1976), for 
example, argue that for speech to add significantly to learning, it must not 
"merely take the place of a hard-copy manual or of printed text on a computer 
terminal.' 1 For example, one method of adding value to the use of audio might 
be to personalize the software. Personalization strategies have been shown to 
increase the effectiveness of CBI (Ross, 1983, Ross & Anand, 1987). 



ERLC 



853 



Digitized Speech as Feedback 
AECT1993 4 



Research on the use of speech in CBI has had mixed results. Research 
conducted with young children learning phonics and vocabulary (Sales & 
Johnson, 1991) found no achievement or attitude differences for students in 
several treatments involving different levels of speech and "motivational" 
sound. The authors suggest several explanations including the low level of 
students' metacognitive skills, and the redundant presentation of critical 
information through text. 

Two research studies examining the different uses of audio in computer- 
based instruction were conducted with special needs students (Chiang, 1983, 
Wiener, 1991). These studies had dramatically different findings. Chiang found 
audio designed to reward and motivate students was actually detrimental to 
learning. He argued that audio served as a distractor, causing students to 
require more study time to achieve the same level of learning. Wiener, 
however, found significant achievement effects when using computer 
generated peech to teach word recognition to mentally handicapped junior 
high school students. 

In summary, speech can be a critical component of instruction. It appears 
to be particularly valuable as a means of communication between the teacher 
and the learner. Much of the communication results from when and how the 
message is delivered. These characteristics of the discourse may be essential 
to the learning outcomes. 

Speech and other auditory elements of CBI have had mixed success in a 
number of research studies. Our limited understanding of these findings is in 
part due to the small amount of research that has been conducted. This, in 
turn, can be traced to the rapid changes technologies and the fact that the 
commercial application of speech in CBI has only recently become practical. 
Study One 

Participants. The subjects who voluntarily participated in this study were 
100 fifth grade students from a suburban Minneapolis elementary school. All 
of the students had regular contact with computer-based instruction prior to 
the study. 

Instructional materials. The software, which provided instruction on 
propaganda techniques used in advertising, is based on the software used by 
Carrier and associates (see Carrier &Williams 1988; Carrier, Davidson and 
Williams, 1985). The new version, was designed to take advantage of the 
graphic and sound capabilities of Macintosh computers with monochrome 
monitors. It was created using HyperCard 2.1 and for this study it was run on 
Macintosh LC computers in a thirty station lab. 

During the data collection, all of the participants were required to wear 
headphones. The software and individual student records were stored at each 
workstation on the hard drive. At the end of each day of data collection, the 
student records were collected and aggregated for analyses. 

Design. A 2x2 design was used (Figure 10) in this research. Variable one, 
degree of familiarity with the feedback provider, has two levels - familiar and 
unfamiliar. The second variable, type of feedback, has two levels - Voice Only 
and Voice with Animated Character. The treatments in the familiar condition 
used a person of authority (the schools media center teacher/librarian) that 
was known to the subjects as the vehicle for delivering feedback . In the 
treatments in the unfamiliar condition a fictitious character unknown to the 
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subjects was used. In addition to the voice feedback, two groups received a 
visual of the speaker's face. Animation techniques were used to create the 
appearance that the speaker was actually talking as the feedback was 
delivered. 

Procedures. Subjects participated in the research in intact classroom 
groups but individuals within groups were randomly assigned to one of four 
treatments. Classroom teachers did not play a role in the delivery of the 
instruction. 

Over a period of three days, each subject completed two computer-based 
lessons on techniques used in advertising. The first lesson, which lasted 
approximately 20 minutes introduced students to four techniques and their 
defining attributes. Students were given an opportunity to practice identifying 
examples and non-examples of two of the techniques introduced. The second 
lesson, which lasted approximately 30 minutes, provided practice on the 
remaining two techniques and allowed for a brief review. Following the second 
lesson the students completed an on-line test consisting of 25 items 
(Cronbach's alpha, r=.87), five representing each of the four techniques studied 
and five non-examples. Finally, students responded on paper to seven 
questions focusing on effort, attitude and learning. 

Data Analysis and Results. Analyses of variance were conducted to 
determine the effects of the treatment variables on the posttest scores. 
Neither the familiarity of the voice nor the presence of a feedback provider 
influenced overall performance (F=.54, p=.462 and F=.99, p=.321, see Table 1). 
No interactions of the treatment variables were found. Gender was found to be 
significant with girls outscoring boys by an average of 2 1/2 points. Means for 
the two groups are shown in Table 2. 

Discussion. Analysis of the data related to the primary research 
questions, which addressed the issues of familiar versus unfamiliar and voice 
versus animated speaker, found no differences. This may indicate that the 
attribution of authority does not transfer to a computer representation of a 
teacher. Or, it may mean that a student's sense of accountability in a 
computer-based lesson is not influenced by the attribution of feedback to a 
specific source. Regardless of the source of the feedback as represented in the 
software, students may simple associate the feedback with the inanimate, 
non-judgmental delivery technology. 

Some issues of secondary interest were identified and are deserving of 
discussion. Use of the "Say it again." option, which allowed students to have 
the verbal feedback repeated, appeared to be related to gender and treatment. 
Boys tended to use this option more in the familiar/voice with text treatment. 
Girls tended to use this option more in the unfamiliar/voice with face 
treatment. 

Another, and perhaps related, secondary finding is that in spite of the 
pattern of option button use, girls spent more time working through 
treatments in which the feedback provider was known. Boys spent more time 
in treatments where the feedback provider was unknown. These findings 
might indicate that girls were willing to work harder at studying the 
information on the screen when they knew the feedback provider. When the 
provider was unknown, they may not have tried as hard. The feedback 
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provider in the software studied was a female. This may also account for the 
differences in the ways in which the boys and girls used the material. 

As a result of these findings a second study was designed and conducted to 
further explore the use of speech as feedback. It was determined that for the 
Sficond study we would use a male speaker was unknown to the learners. Two 
new treatments, voice only and text thac was read, were added to isolate any 
offect attributed to the voice variable. 
Study Two 

Participants. Subjects volunteering to participate in this study were 145 
sixth grade students in a suburban Minneapolis middle school that served only 
fifth and sixth grades. Subset of volunteers with high (N=41) or low (N=49) 
reading abilities were identified. Only the scores from these individuals were 
used for this study although all 145 participated in the instruction. All of the 
participants had had regular contact with computer-based instruction prior to 
this study. 

Instructional Materials. The instructional materials used in this lesson 
were the same as those describe in Study 1 except for the delivery of feedback 
which was specific to the treatment condition. The feedback in this study 
differed in that the audio was a male voice and the animated figure was of an 
unfamiliar male cartoon character. 

The computers used in this study were in two labs of approximately 15 
Macintoshes each. One lab was equipped with SE30 computers, the other was 
equipped with LC computers. 

Design. A 2x3 research design was used. Variable one, type of feedback, 
has three levels - (1) spoken audio only, (2) printed and read (spoken), and (3) 
spoken by an animated character. The second variable, reading ability, has 
two levels - high and low. 

Procedures. Subjects* classroom groups were divided in half and each half 
was required to report to one of two Macintosh computer labs where the 
studies were conducted. In each lab the subjects were randomly assigned to 
one of the treatments. Classroom teachers did not play a role in the delivery of 
the instruction. 

Data Analysis and Results. Analyses of variance were conducted to 
determine the effects of the treatment variables on the posttest scores. Type 
of feedback was not found to have a significant effect on students 1 performance 
in this study (Table 3). However, ANOVA results do indicate that reading was 
significant (Table 4). 

Discussion. Analysis of the data related to the primary research question 
(type of feedback) found no difference. However, there was a consistent 
tendency for students receiving only voice feedback to score higher on the 
posttest. This may indicate that supplementing the audio with text or 
graphics divides the learners* attention resulting in decreased comprehension. 

The findir *s related to reading ability appear to reflect the participants' 
ability to read und learn from the text in the lesson. These findings indicate 
that the audio feedback component does not compensate for the text 
presentation of the content. 
Conclusions and Recommendations 

Following are a number of conclusions and recommendations based on the 
results of these studies and the observations of the researchers. While they 
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may be tentative, they none-the-less provide food for thought and direction for 
future work. 

1. Whose voice is used in the delivery of feedback in CBI seems to be of little 
importance. Perhaps of more importance is the gender of the speaker. In 
our first study, where a female agent was used, female students 
outperformed male students. In our second study, where the agent was a 
male, no performance difference based on gender was found. Whether this 
indicates that (1) girls work harder for a female agent, or (2) boys don't 
work as hard for a female agent, is worthy of further investigation. 

2. The imago associated with the speaker ie of little importance. The 
students in our study did not appear to associate an agent on the screen 
with the person being represented. A possible topic for further 
investigation is whether younger learners would attribute more authority 
to the computer agent that represented a person they know. 

3. It is reasonable to assume that if speech is going to effect a change in 
learning outcomes, then the amount and positioning of the speech within a 
lesson are of critical importance. In the studies we conducted speech was 
used only to deliver feedback. Much of the content was delivered through 
text. In our second study students 1 reading ability was found to have a 
significant effect on their learning. Therefore, it is reasonable to assume 
that an effective use of speech technology would be to help students 
overcome reading limitations that influence their processing of content. 

4. During the research teachers and school staff repeatedly commented on 
the effect headphones had on student behavior. Once the headphones 
were on, student focused their attention on the computer lesson. Normal 
computer lab behavior appears to involve talking with others and looking 
around the room. Whether this auditory isolation produces better 
concentration, and thus improved performance, is uncertain. 

5. Another observation made during the studies was that students in the 
animated character treatments were more demonstrative. They showed 
more facial expressions and talked back to the computers more than 
students in other treatments. The observers felt these students seemed 
to be more involved and to enjoy the lessons more. This may indicate a 
particular value to this method of integrating speech. 

6. Regardless of what our research finds,we may not have a choice but to 
use speech and agents in instructional software. The use of these 
techniques is similar to the use of color. Increasingly our clients expect 
products to contain certain features. When they are not present, 
questions are raised and instructional products are devalued. Perhaps the 
most we can hope to do at this time is to moderate what we believe we 
can accomplish through the use of speech technology and proceed 
cautiously with its use. 
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7. In multimedia environments identification of primary symbol systems 
and the "most essential characteristics" of each is no small tesk. For 
spoken words, the list of characteristics includes, but is not limited to: 
gender of the speaker, tone, mode, pacing, cultural cues, and message. 
Furthermore, these characteristics may have varying degrees of 
importance to different learners. 
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ADVERTISING 



Be 



While you are working on thte tesson 
Ms. Burton be telling you how you are 
doing. Be sure to wear your 
earphones I 



W you want to bear Ms. Burton say 
something, Just press the 'Say tt 
again./ button. Try tt now. 



When you have finished the tesson, you 
win be given a test to check how much 
you teamed about these four 
techniques. 




Figure 1. Introduction from Propaganda Techniques 
in Advertising lesson describing audio controls. 



ADVERTISING 



©AND WACOM - 7 





Here's a chance for you to practice 
spotting when the Barxiwagon 
technique 1$ used In ads. 
Ms. Strandberg will help you. 

When you are asked a question, cWdcc 
on the best answer that appears here. 

Then cfeck here to check you answer 



Figure 2. Beginning of practice section explaining rr 
sponse options. 



flTI SING BANDWAGON 11 



Now try this one: 



Manncsotans 
choose MARY'S 
CHERRY PIE 
every time! 




Is ttts advertisement an example of the 
BancNvagon technique? 



is 




111 



Figure 3. Typical practice item from the "familiar" 
voice and actor treatment just prior to response 

checking. 



ADVERTISING bandwagon 11 



Nowtrythteone: 



l&anesotans 
choose MARY'S 

CHERRY PlEs<^& 



Words fika *Minne$otan$, moms, 
and kids' refer to a LOT of people. 
You are persuaded to buy 
something just because a tot of 
these people do. So tf its ad does 
use the Bandwagon technique. 



IE 




Figure 4. Additional feedback appearing in ar. 
window after audio feedback is delivered. 



ADVERTISING 



Now try this one. 



ft 



Youcjp' 
rx: K . one 



mm «hmcfttr«ittK 



The taty to the correct answer ts that 
the famous person b only shown with 
the drink that te advertised. He fas 
NOTleilng you to drtnkthe product 
Thcs IS an eicampte of Transfer. 




Him 



Figure 5. Feedback as displayed to the voice only 
treatments (both "familiar" and "unfamiliar" voice). 



ADVERTISING transfer - 10 



fcaianrw Win (Champion) art DIET COKE 




Does tt use the Transfer technique? 




Figure 6. Practice item using the "unfamiliar" voice 
and actor. 
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ADVERTISING TtSTwraHua-s 



TrythJsonc. Does thb ad use the 
TEST! MOMAL technique? 



* We love 
Oscar Mayer 
hot dogs!" 




*4<j|tt«gefai_ 




Figure 7: Spoken only- 



ADVERTISING bandwagon - e ; 



Look attt^ad: 



'Wgk- KIDSLOVE 
^» RAISINS, 




' There is txhhiog artificial 
in s raxsio's little body. 



sags*-: 



Is mis anex2mpte of Bandwagon? 




:rc- 6: Printed and read (scoken). 



Try this or* 



BAKDVASOH-9 



! ! 




Last year. NAPA sold two auto \ | 

parts for every man, woman, 
I child, . .and dog in America, 

Does this cxL use the Bandwagon 
technique? 




'icure 9: Sooken bv an ar:i~i«ec character. 



862 I 



BEST CiiPV AV'AtLASU 



Digitized Speech as Feedback 
AECT 1993 13 



Table 1 

Analysis of Variance - Tests of Significance for Posttest Scores 



Source of Variation 


MS 


F 


Sig of 


\i/ifhin ppIIq 
WlUliil V/t/ll3 


25.88 






covariate 


785.20 


30.34 


.*9 


familiar/unfamiliar 


14.10 


.54 


.46 


voice/image 


25.71 


.99 


.32 


gender 


176.72 


6.83 


.01 


familiar/unfamiliar by voice/image 


.64 


.02 


.87 


familiar/unfamiliar by gender 


18.88 


.73 


.39 


voice/image by gender 


1.35 


.05 


.82 


familiar/unfamiliar by voice/image by 


3.34 


.13 


.72 



gender 



Table 2 

Summary of Posttest Scores by Gender 



Gender 
Boys 

Girls 



Mean 
Count 

Standard Deviation 




1 


12.68 




47 




5.56 


2 






15.26 




53 




5.89 



14.05 
100 
5.85 
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Figure 10 

2x2 Research Design, Cell Size and Gender 





Familiar 


Unfamiliar 


Voice 


22 


24 


(with text) 


(14 girls, 8 boys) 


(17 girls, 7 boys) 


Voice with 


29 


25 


Animated Character 


(11 girls, 18 boys) 


(11 girls, 14 boys) 
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Table 3. Means and Standard Deviations for Posttest Scores (Treatment by Ability) 



Treatments 

Ability 1 2 3 Total 



Low 



High 

Mean: 21.87 22.11 21.59 21.80 

SD: 3.83 2.57 4.00 3.59 

N: 15 9 17 41 

Mean: 16.07 13.95 15.13 14.94 

SD: 4.53 6.73 5.57 5.74 

N: 14 19 16 49 

Combined 

Mean: 19.07 16.57 18.45 18.07 

SD: 5.06 6.87 5.77 5.95 

N: 29 28 33 90 



Table 4. Two-way ANOVA Results for Posttest Scores (Treatment by Reading Ability) 



Source 


df 


SS 


MS 


F 


P 


Treatment 


2 


12.38 


6.19 


0.25 


.778 


Ability 


1 


985.60 


985.60 


40.14 


.000 


Interaction 


2 


19.47 


9.73 


0.40 


.674 


Error 


84 


2062.37 


24.55 
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ERIC 865 



